Bug report: https://dagster.slack.com/archives/C04CW71AGBW/p1700237771181059?thread_ts=1697638899.590569&cid=C04CW71AGBW

Synopsis:

• At definition time, we evaluate @dbt_assets using DBT_INDIRECT_SELECTION=eager [side note, I'm kind of surprised this isn't configurable].
• At runtime, if you're using checks, we set DBT_INDIRECT_SELECTION=empty so that we have the option to exclude checks if they're not in the selection
• If we are in a subset, this works because we explicitly pass the selected models and tests
• However, if we determine we're not in a subset, we pass the default selection string, which may not select the tests now that we're not using eager.

Solutions:

1. [this diff] If asset checks are enabled, always pass an explicit selection of models and tests
2. The reason we try to use the default selection string when we can is to make things readable from the dbt side. We could try to preserve that by only setting DBT_INDIRECT_SELECTION=empty when we're doing a subset and passing the explicit selection, otherwise use eager and the default selection.

A concern for (2) is relationship tests. Take the case of two models

• customers, one non null test
• orders, one relationship test with customers

Dbt will resolve selection customers to the customers model and both the null test and the relationship test. If you're not using asset checks this is fine, we'll emit observations for both tests. If you are using checks, currently @dbt_assets(...select="customers") will only define the null test. This makes sense- we wouldn't even display the relationship test if we ingested it, because we didn't ingest the model its on.

With #1, we won't execute the relationship test (which matches the current behavior). With #2 we will whenever we switch over to eager, which is unexpected because we didn't model it.

Side note: we should document that when you enable checks, relationship tests on non selected models will no longer run